IMPORTANT:

- As discussed earlier with you on whatsapp, most of the queries on Hive
failed because of a Yarn memory issue. (Screenshot attached = error query.jpeg)


- After doing an intensive research on the internet we concluded that the
dataset we're trying to query is too big and hence it overfloods yarn's memory.

Solutions we tried to fix this:

1) Completely re-running the entire process and also cleaning yarn's memory and then
restarting it.

2) We tried running queries by changing the Hive execution engine from 
MapReduce to tez using:

set hive.execution.engine = tez 

but the issue still persisted.

3) Thought of proceeding with Track 1 instead but soon realised that there's not enough
time and we've already spent too much on this. 



Reason for our large dataset:
- We're working on hadoop and there's no point in generating and querying data consisting
of only several thousand rows. Unlike our classmates who successfully ran queries 
on a smaller sized dataset, we generated 50 lacs+ rows because the idea was to analyze
a dataset pertaining to a larger group.

- It was absolutely inevitable for the kind of project we received to have a large dataset since 
we're collecting data from numerous sensors on short and fixed intervals.


